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Abstract 

Incidental vocabulary learning has attracted a great deal of attention in ELT research. However, it 
is important that teacher and researcher exploitation of vocabulary developments be guided by 
more than replication of previous research designs. For conclusions based on empirical research to 
be valid, it is important to be clear about exactly what any data being gathered pertains to. While 
Karaka§ & Sarigoban (2012) claim to have presented a solid piece of research on the effects of 
subtitled cartoons on incidental vocabulary learning, in practice it is not so. It is argued that the 
research design validity resulted in questionable results having little relevance to genuine 
incidental vocabulary learning. 

There has been a line of investigation in the recent years that replicates dominating faulty 
research designs and which leads to reports of little or no significant results on incidental 
vocabulary learning. In this commentary I will offer my criticism of the research stand that is 
taken by Karaka§ & Sarigoban (2012), arguing that a) a one-time treatment is a discrepancy to 
the understanding of incidental vocabulary learning processes and b) findings are to be attributed 
to another area of research; these two considerations arise mainly from concept 
misinterpretation. Further in this commentary I will also elaborate on inaccurately reported data. 

The aim of the research carried out by Karaka§ & Sarigoban (2012) was to find out 
whether watching subtitled cartoons influenced incidental vocabulary learning. Particularly, the 
research aims to answer the question “How would a language learning material (i.e, animated 
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cartoons) with or without English language subtitles affect the students’ vocabulary 
development” (p.4). 

In the literature review, Karaka§ & Sarigoban (2012) rightfully discuss incidental 
vocabulary learning, thus: 

Vocabulary, the core of the language, is not acquired at one shot. It necessitates a long process. 

Throughout this process, learners become familiarized with the encountered words. What makes 

them familiar with words for acquisition is the frequency of their usage and the number of 

encounters in different forms and contexts (Nation, 1990; Schmidt, 2001). 

Karaka§ & Sarigoban (2012, pp.5-6) 

This idea is repeated in the introduction, voiced as follows: “the view that vocabulary 
acquisition is a continuum of development” (p.4) and in the conclusion “Vocabulary 
development is a long lasting process ...” (p. 12). However, Karaka§ and Sarigoban seem to 
assume that such incidental picking-up of vocabulary will occur only after one exposure to the 
items being tested, as evident by the research design of the study. 

Critique 1: One-time exposure 

To reiterate, incidental vocabulary learning/acquisition is the learning of new words as a by¬ 
product of a meaning-focused communicative activity , such as reading, listening, and interaction. 
It occurs through "multiple exposures to a word in different contexts" (Huckin & Coady, 1999, p. 
185). Waring and Takaki (2003) have also made the same point. Again, Waring and Nation 
(2004, p.16) conclude that “incidental learning of vocabulary is best considered as a cumulative 
process where learners build up knowledge of a word through repeated encounters over a 
reasonable period of time" (my emphasis). 

“Effective incidental vocabulary learning is a conscious learning process” (Laufer & Hill, 
2000) in that it requires attention on the part of the learner on word features while attempting to 
infer the word meaning from the context. This involves deep processing of information; yet deep 
processing during the first encounter is unlikely to induce long-term retention. Laufer and Hill 
(2000) point at evidence surveyed by Nation (1990) that “repeated exposures to a new word in 
language input reinforce learning, though it is unclear how many repetitions are necessary for 
this. Sailing (1959) suggests that the number is 5, Kachroo (1962), Crothes and Suppes (1967) 
suggest it is 7, Saragi, Nation, and Meister (1978), 16”. Since incidental vocabulary acquisition 
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takes place incrementally over a period of time, and there is no agreement as to how many and 
what kinds of exposures are needed for successful acquisition, the research measuring one 
exposure to the target words represents an extremely narrow understanding of the notion of 
incidental vocabulary acquisition, if at all correct. Therefore, I argue that a single exposure to the 
target words cannot be a reliable measure to conclude about effectiveness in incidental 
vocabulary learning. 

Another point I would like to make in support of my critique is the procedure. Karaka§ 
and Sarigoban decided upon the procedure of following similar studies with 1 or 2-week spans. 
Here the researchers simply replicated previous research frameworks without a critical 
consideration and without doubts that such frameworks may be faulty. The pre-test / post-test 
approach may bring about maturation issues, as Karaka§ and Sarigoban themselves acknowledge: 
“the interval between the two tests was too short, which might have caused bias in the 
interpretation of the results” (p.13) 

The treatment was done during class time and the participants did not know they would 
be tested. This may be considered both an asset and a disadvantage at the same time. Incidental 
vocabulary learning, being "picking up" words as a by-product of a communicative activity, is a 
matter of the learner’s individual choice whether to remember a word or not. Not knowing they 
were to be tested, the participants might have made the choice to only ‘do the task’ focusing on 
the effort to infer the meaning rather than attempt to remember or learn a word. On the other 
hand, the very fact that a word is a target word in the pre-test may have had learners choose to 
‘remember’ a word during the pre-test and later retrieve that word during the post-test only 
recognizing the graphemic representation. 

On the whole, incidental vocabulary acquisition may not always occur and there is no 
control over what is to be learned, though it may seem too bold a statement on my part. 

The next point I will bring into discussion is word choice and word frequency. The 
authors offer no justification for the choice of words (verbs in this case) - why verbs, why both 
concrete and abstract verbs? 

Concerning word frequency, the reader could not find a justification for the researchers’ 
choice. There’s evidence from research that word frequency affects incidental vocabulary 
learning and retention. For instance, Anne Ferrell Tekmen and Daloglu (2006) examined the 
relationship between learners’ incidental vocabulary acquisition and word frequency in a text and 
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reported that word frequency was a significant factor in vocabulary acquisition (p < .05), with 
29% of the variance in acquisition being accounted for by frequency. The aforementioned may 
well have played a factor in the treatment in Karaka§ and Sarigoban’s study. 

Further on, the data collection tool used in the study is Wesche and Paribakht’s 
Vocabulary Knowledge Scales (VKS). Karaka§ and Sarigoban’s justification for their choice of 
research tool is as follows: the main aim was to identify the initial stages or levels in 

vocabulary in vocabulary developments by students’ self-reports and demonstrations” (p. 12). 
These words, in the first place, are a quotation of Wesche and Paribakht in Meara’s discussion 
paper, thus “[the aim of the Scales is] to capture the initial stages of levels in word learning 
which are subject to self-report or efficient demonstration, and which are precise enough to be 
used to reflect gains during a relatively brief instructional periods...” (Meara, 1996, p.6). 
Secondly, the words are interpreted by Karaka§ and Sarigoban out of the wider context of 
Meara’s overall discussion. Meara claims that both Richards’ (1976) word knowledge 
framework and Wesche and Paribakht’s Vocabulary Knowledge Scales approach vocabulary 
from the perspective of individual words. Instead, he argues that researchers should not deal with 
individual words but whole lexicons, and proposes a multistate model which Meara and 
Rodriguez Sanchez (1993) have used to successfully “predict long term distribution of words in a 
large target vocabulary across four states of word knowledge” (Meara, 1996, p.8) - long term for 
them meant 46 months. 

Karaka§ and Sarigoban’s investigation deals with individual words and the VKS seem 
appropriate for such a purpose. There is a discrepancy, however, in the understanding that, as 
Meara points out, the VKS represent a progression from 1 to 5 and “there is no reason to believe 
that the VKS descriptors reflect a succession of stages” (Meara, 1996, p. 6). I fully support this 
statement, because it is perfectly possible for learners to reproduce a sentence with a fixed 
expression which contains a target word (Level Five on the VKS) while at the same time not 
knowing the meaning of the word (Level One on the VKS). That is to say the “levels” can 
coexist simultaneously suggesting a flat structure of the Scales rather than a hierarchical one, 
where knowing a word at a higher level is thought to be more fully integrated into the learner’s 
language. Further on, Meara (1996, p.7) suggests the five levels be interpreted as stages and that 
words can be in any of the five states at different times while words are being integrated into the 
lexicon of the learner. Karaka§ and Saricoban’s understanding of word knowledge integration 



Teaching English with Technology, 13(2), 75-82, http://www.tewtjoumal.org 


79 


seems to be reflected in the following statement: “[the VKS] allows to specify the stages of 
vocabulary acquisition from first exposure to production and enables the researcher to determine 
how well the participants know these vocabulary items” (p.8). The researchers seem to assume 
that one exposure leads to either not knowing the word (Level One - "I don’t remember having 
seen this word before”) or knowing a word in one of the other four states without accounting for 
the simultaneous coexistence of the levels/stages. 

The other problematic aspect concerns the rater’s downgrading the learners wrong 
choices on the scales by one level, which also reflects the researchers’ misconception that words 
cannot coexist and/or words at higher levels are better integrated than words at lower levels. 

Critique 2: Misinterpretation of results 

The second major point of discussion in this commentary relates to the interpretation of the 
results. I argue that the results should not be interpreted in the light of incidental vocabulary 
learning. Viewing comprehension is greatly aided by non-verbal cues in while-watching learning 
activities. Therefore, the results should be attributed to the research on inferring meaning from 
context rather than incidental vocabulary learning. The reported significant improvement in the 
pre-test / post-test results confirms, in reality, short-term vocabulary gains. The process was 
attributed to contextualization, i.e. inference of meaning from context. Karaka§ and Saritjoban 
themselves claim “the incidental learning of the vocabulary items occurred due to the 
incorporation of target words into the cartoons that functioned as a context...” (pp.11-12). Their 
statement that “[t]he actions, signals of hands and arms, as well as facial expressions might 
facilitate the understanding of the target verbs [...]” (p.12) can only confirm my position in this 
commentary that the findings could better illustrate short-term vocabulary gains. 

Critique 3: Inaccurately reported data 

Trying to replicate the study I was confronted with the impossibility of doing so. I was interested 
in finding out the word frequency of the eighteen items tested by Karaka§ and Sari§oban and got 
hold of the cartoons from Season 3 with their English language subtitles, namely Episode 3 - Mr. 
Griffin goes to Washington and Episode 6 - Death Lives. I used a simple search functionality to 
determine the word frequency in the episodes. I present the results in the table below. If anyone 
is interested in the verification of the results I gathered, the two subtitle files which I examined 
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are accessible online in .srt format on http://ewbooks.info/critique/Season3 Episode03.srt and 
htt p://ewbooks.info/critique/Season3 Episode06.srt . 


Table 1. Target word frequency in Episodes 3 and 6, Season 3, Family Guy 



Target 

word 

N° of occurrences plus + forms in Episode 3 

N° of occurrences + word forms in Episode 6 

1 

mop 

- 

2x mop 

3x mopping 

2 

hug 

- 

lx hug 

3 

hail 

- 

- 

4 

sigh 

- 

- 

5 

fart 

- 

- 

6 

gasp 

- 

- 

7 

swab 

- 

lx swab 

“Swab means to mop” 

8 

dump 

- 

2x dump 

9 

rely 

lx rely 

“In which we used to rely?” 

lx rely 

“In which we used to rely?” 

10 

ditch 

- 

lx ‘m ditching 

11 

growl 

- 

- 

12 

vomit 

- 

- 

13 

roast 

lx roasted coffee (adj.) 

- 

14 

murmur 

- 

- 

15 

chuckle 

- 

- 

16 

applaud 

lx applaud 

- 

17 

negotiate 

lx negotiate 

- 

18 

terminate 

lx terminated 

- 


Strikingly, eight of the target words do not appear in the subtitles in the two episodes. While 
these actions are performed by the characters in the cartoons (words 3, 4, 5, 6, 11, 12, 14, 15), 
the eight words are neither spoken in the voiceovers nor present in the subtitles. It is only natural 
to posit the question “How are then learners in the subtitles Group A tested on words they had 
not seen written or heard while watching the cartoons with the English subtitles?” It seems the 
students are required to understand and learn English words without seeing the written form and 
without hearing the spoken form; the meanings of these words are to be understood relying on 
extra linguistic features. 

Table 1 also illustrates that seven other words occur only once in both subtitle files, 
which brings back the researchers’ assumption that one-time occurrence will be enough for a 
word to be learned and that a single exposure to the target word will yield reliable results to 
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conclude about incidental vocabulary learning, which I reiterate, is understood by Karaka§ & 
Sarigoban as “[v]ocabulary, the core of the language, is not acquired at one shot..” (2012, p.5). 

Moreover, some of the words in the subtitles do not occur in the form of the target words 
(words 1, 10, 13, 18). Knowing a word and its forms requires deeper processing on the part of 
the learner. If for instance the word to test is “roast” (as the researchers say they test only verbs), 
the word that is presented in the subtitles and spoken in the cartoon is an adjective. In my view 
this is ill-designed test administration. The results that Karaka§ and Sarigoban report are 
unreliable, not to say that this might be downright fabrication. To me all the above renders the 
whole research and all the findings invalid. 

Conclusion 

In conclusion, incidental vocabulary learning is defined as multiple representations in different 
contexts over a significant amount of time and research designs that measure single exposure to 
target vocabulary cannot yield reliable data. Therefore I call for the discontinuation of such a line 
of investigation. 

The research paper in focus by Karaka§ and Sarigoban defines incidental vocabulary 
learning as “multiple representations of a word in different contexts” but their research design is 
based on one-time exposure. 

Secondly, the results obtained by Karaka§ and Sarigoban are attributed to incidental 
vocabulary learning, while the researchers contradict themselves by claiming the target words 
were learned by the students from the context, not by the treatment. The results, I argue, should 
be attributed to short-term vocabulary gains. 

Thirdly and most importantly, Karaka§ and Sarigoban reported eighteen target words 
while only ten of them are present in their spoken and written forms in the subtitles. 

On the whole, however solid Karaka§ and Sarigoban’s investigation seems on the surface, 
the findings are questionable and unreliable. Reporting false data is to say the least misleading 
and unethical. 
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